Adaptive input pen mode selection

ABSTRACT

An adaptive pen mode selection is included within a pen input based computing system. The adaptive pen mode selection automatically switches the operating mode from an input data mode to the data item selection mode when a pen stroke is made over an existing data item to permit it to be selected for editing, deletion and replacement. The adaptive pen mode selection provides an improved user interface returns the computer to the user specified operating mode when the selection operation has been completed to allow the user to continue entering data in the user specified operating mode. With adaptive pen mode selection, a user of a pen input based computing system does not need to change the operating mode of the computer to an data item selection mode using the control inputs in order to select one or more data items for editing.

This application is a continuation of application Ser. No. 09/550,362,filed Apr. 19, 2000 now U.S. Pat. No. 6,683,600, which application(s)are incorporated herein by reference.

TECHNICAL FIELD

This invention relates in general to a method, apparatus, and article ofmanufacture for providing data input interface for a pen-based computingsystem, and more particularly to a method, apparatus, and article ofmanufacture for providing an adaptive input pen mode selection toautomatically select existing data items in support of editingoperations.

BACKGROUND

Pen input based computing systems are becoming increasingly prevalentwith the creation of hand-held computing system. When a user enters datainto a pen input based computing system, typically the hand-heldcomputer operates in one of a number of input operating modes. Theseoperating modes may include an ink pen input mode, a text input mode,and a data item selection mode. The operation of the hand-held computermay differ greatly depending upon the operating mode in which thecomputer is currently set.

When the computer is operating in the ink pen input mode, pen strokestypically are treated as entering graphical pen strokes onto a datawindow in a similar manner to the drawing of a line onto a sheet ofpaper. When the computer is operating in the text input mode, the usertypically uses a soft input window to enter text characters as ifprinting or typing at a keyboard. Finally when a computer is operatingin the data item selection mode, the pen strokes are interpreted as aninstruction to select the data items over which the pen stroke is made.

A user switches between these operating modes by indicating whichoperating mode is desired using a control input. Control inputs aretypically used to specify any number of operational parameters such asinput mode operation of a pen input based computing system. Thesesystems typically require the user to constantly change the operatingmode between the possible operating modes using the control inputs. Assuch, a user may interact with the control inputs to constantly switchthe operating mode of the user interface between every pen stroke if acomplex set of data items is to be edited.

Typically, a user may toggle between one input mode and the data itemselection mode as data items are edited, deleted, and replaced. Thisrequirement that the user continually provide input commands to switchthe operating mode of the pen input based computer as part of an editingoperation decreases the user efficiency in entering data and editingexisting data items.

SUMMARY

In accordance with the present invention, the above and other problemsare solved by providing an adaptive pen mode selection thatautomatically switches the operating mode from an input data mode to thedata item selection mode when a pen stroke is made over an existing dataitem to permit it to be selected for editing, deletion and replacement.The adaptive pen mode selection provides an improved user interface byreturning the user to a previously specified operating mode when theselection operation has been completed in order to allow the user tocontinue entering data in the user specified operating mode.

The great utility of the invention is that the user of a pen input basedcomputing system, such as a hand-held computer, does not need to changethe operating mode of the computer to a data item selection mode usingthe control inputs in order to select one or more data items forediting. This utility will improve the user efficiency in operating thepen input based computing system.

One aspect of the present invention is a method of providing an adaptivepen mode selection within a pen input based computing system having oneor more user selected operating modes including an ink pen input mode, atext input mode, and a data item selection mode. The method comprisesdetecting a pen stroke having a location within a window on a display ofthe computing system, determining the location of the pen stroke uponthe display of the computing system, and determining a location of allexisting data items displayed within the window containing the locationof the pen press. If the computer is operating in the data itemselection mode, the method selects all data items having location thatcorresponds to the location of the pen stroke. If the computer is notoperating in the data item selection mode, the method determines whetherthe user desired to select all data items having location thatcorresponds to the location of the pen stroke and selects all data itemsdetermined to be intended to be selected by the pen stroke.

Another aspect of the present invention is a computer program productreadable by a computing system and encoding a set of computerinstructions for providing an adaptive pen mode selection within a peninput based computing system. The computing system has one or more userselected operating mode including an ink pen input mode, a text inputmode, and a data item selection mode. The computer data productcomprises instructions that detect a pen stroke having a location withina window on a display of the computing system, determine the location ofthe pen stroke upon the display of the computing system, and determine alocation of all existing data items displayed within the windowcontaining the location of the pen press. If the computer is operatingin the data item selection mode, the computer data product instructs thecomputing system to select all data items having location thatcorresponds to the location of the pen stroke. If the computer is notoperating in the data item selection mode, the computer data productinstructs the computing system to determine whether the user intends toselect all data items having location that corresponds to the locationof the pen stroke and select all data items determined to be intended tobe selected by the pen stroke. The pen stroke possesses propertiesincluding the location including a starting location, and endinglocation, a direction, and one or more segments of constant slope. Theproperties of the pen stroke are used in determining whether the userintends to select all data items having location that corresponds to thelocation of the pen stroke. The existing data items possess propertiesincluding a data type, a position, a size, and a selection location, anda persistence age. And the properties of the existing data items areused in determining whether the user intends to select all data itemshaving location that corresponds to the location of the pen stroke.

Yet another aspect of the present invention is a pen input basedcomputing system having adaptive pen mode selection within one or moreuser selected operating mode including an ink pen input mode, a textinput mode, and a data item selection mode. The computing systemcomprises a programmable processor, a user display configured togenerate pen strokes corresponding to user created pen movements acrossthe display, one or more data items having properties including a datatype, a position, a size, and a selection location, and a persistenceage, an user interface module for capturing user generated pen strokes,and a data item input module for determining whether data items are toselected by a pen stroke. The pen stroke possesses properties includingthe location including a starting location, and ending location, adirection, and one or more segments of constant slope. And the data iteminput module determines whether the user intends to select all dataitems having location that corresponds to the location of the pen strokeand selects all data items determined to be intended to be selected bythe pen stroke.

These and various other features as well as advantages, whichcharacterize the present invention, will be apparent from a reading ofthe following detailed description and a review of the associateddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a hand-held computer having text data items and handwritten data items with a data item highlighted during editing accordingto one embodiment of the present invention.

FIG. 2 illustrates a general purpose computing system for use as onemore computing systems according to another embodiment of the presentinvention.

FIG. 3 illustrates a hand-held computer having processing modules toimplement a pen-based user interface according to another embodiment ofthe present invention.

FIG. 4 illustrates a hand-held computing system having multiple dataitems in which data items may be selected according to one embodiment ofthe present invention.

FIG. 5 illustrates a data item input module according to one possibleembodiment of the present invention.

FIGS. 6 a–c illustrates various data items and corresponding pen strokesaccording to one possible embodiment of the present invention.

FIG. 7 illustrates an operational flow for an adaptive pen modeselection process according to an example embodiment of the presentinvention.

DETAILED DESCRIPTION

FIG. 1 illustrates a hand-held computer having text data items and handwritten data items with a data item highlighted during editing accordingto one embodiment of the present invention. The hand held or palm-sizedcomputer 101 is used with an input pen 102 to enter data items 111–113by writing on the screen 114. These data items may include user createditems 111, hand written items 112, and various text items 113. All ofthese data items may be edited by selecting all or part of a data item114 so that the data item 114 can be edited and/or deleted.

The user of the hand held computer 101 typically selects the data itemfor editing by dragging the cursor over the desired data item 114. Whenthe user drags the cursor over a data item 114, the adaptive pen modeselection will correctly determine whether the user wants the data itemhighlighted for editing or whether the user wants a line drawn throughthe data item. The adaptive pen mode selection operations include aninteraction between various user interface modules and various pen modeinput modules.

The hand-held computer 101 operates in one of a number of inputoperating modes. These operating modes may include an ink pen inputmode, a text input mode, and a data item selection mode. In the ink peninput mode, pen strokes typically are treated as entering graphical penstrokes onto a data window in a similar manner to the drawing of a lineonto a sheet of paper. In the text input mode, the user typically uses asoft input window to enter text characters as if printing or typing at akeyboard. In the data item selection mode, the pen strokes areinterpreted as an instruction to select the data items over which thepen stroke is made.

A user switches between these operating modes by indicating whichoperating mode is desired using control inputs 120–122. When a userpresses the pen 120 on top of one of these control inputs 120–122, thecontrol input is highlighted, and the hand-held computer is placed intothe corresponding operating mode. The control inputs will select a dataitem selection mode 120, an ink pen input mode 121, and a text inputmode 122. These input controls may include also selection buttons suchas radio buttons that a user can activate or deactivate as desired.These controls may also include pop-up menu options that are selectedusing a pointing device such as a pen 102 to set the current operatingmode. Any other user interface command input method may be used toselect the operating mode for the computer without deviating from thespirit and scope of the present invention.

FIG. 2 illustrates a computing system that may be used as the hand-heldcomputing system according to one embodiment of the present invention.An exemplary computing system for embodiments of the invention includesa general purpose-computing device in the form of a conventionalcomputer system 200 capable of accepting pen-based input data. Thesystem includes a processor unit 202, a system memory 204, and a systembus 206 that couples various system components including the systemmemory 204 to the processor unit 200. The system bus 206 may be any ofseveral types of bus structures including a memory bus, a peripheral busand a local bus using any of a variety of bus architectures. The systemmemory includes read only memory (ROM) 208 and random access memory(RAM) 210. A basic input/output system 212 (BIOS), which contains basicroutines that help transfer information between elements within thecomputer system 200, is stored in ROM 208.

The computer system 200 further includes a hard disk drive 212 forreading from and writing to a hard disk, a magnetic disk drive 214 forreading from or writing to a removable magnetic disk 216, and an opticaldisk drive 218 for reading from or writing to a removable optical disk219 such as a CD ROM, DVD, or other optical media. The hard disk drive212, magnetic disk drive 214, and optical disk drive 218 are connectedto the system bus 206 by a hard disk drive interface 220, a magneticdisk drive interface 222, and an optical drive interface 224,respectively. The drives and their associated computer-readable mediaprovide nonvolatile storage of computer readable instructions, datastructures, programs, and other data for the computer system 200.

Although the exemplary environment described herein may employ a harddisk, a removable magnetic disk 316, and a removable optical disk 219,other types of computer readable media capable of storing data can beused in the exemplary system. Examples of these other types ofcomputer-readable mediums that can be used in the exemplary operatingenvironment include magnetic cassettes, flash memory cards, digitalvideo disks, Bernoulli cartridges, random access memories (RAMs), andread only memories (ROMs).

A number of program modules may be stored on the hard disk, magneticdisk 216, optical disk 219, ROM 208 or RAM 210, including an operatingsystem 226, one or more application programs 228, other program modules230, and program data 232. A user may enter commands and informationinto the computer system 200 through input devices such as a keyboard234 and mouse 236 or other pointing device. Examples of other inputdevices may include a microphone, joystick, game pad, satellite dish,and scanner. These and other input devices are often connected to theprocessing unit 202 through a serial port interface 240 that is coupledto the system bus 206. Nevertheless, these input devices also may beconnected by other interfaces, such as a parallel port, game port, or auniversal serial bus (USB). An LCD display 242 or other type of displaydevice is also connected to the system bus 206 via an interface, such asa video adapter 244. In addition to the display 242, computer systemstypically include other peripheral output to devices (not shown), suchas speakers and printers.

The computer system 200 may operate in a networked environment usinglogical connections to one or more remote computers, such as a remotecomputer 246. The remote computer 246 may be a computer system, aserver, a router, a network PC, a peer device or other common networknode, and typically includes many or all of the elements described aboverelative to the computer system 200. The network connections include alocal area network (LAN) 248 and a wide area network (WAN) 250. Suchnetworking environments are commonplace in offices, enterprise-widecomputer networks, intranets, and the Internet.

When used in a LAN networking environment, the computer system 200 isconnected to the local network 248 through a network interface oradapter 252. When used in a WAN networking environment, the computersystem 200 typically includes a modem 254 or other means forestablishing communications over the wide area network 250, such as theInternet. The modem 254, which may be internal or external, is connectedto the system bus 206 via the serial port interface 240. In a networkedenvironment, program modules depicted relative to the computer system200, or portions thereof, may be stored in the remote memory storagedevice. It will be appreciated that the network connections shown areexemplary and other means of establishing a communications link betweenthe computers may be used.

The embodiments of the invention described herein are implemented aslogical operations in a hand-held or palm-sized computing system capableof accepting pen-based input data. The logical operations areimplemented (1) as a sequence of computer implemented steps or programmodules running on a computer system and (2) as interconnected logic orhardware modules running within the computing system. Thisimplementation is a matter of choice dependent on the performancerequirements of the computing system implementing the invention.Accordingly, the logical operations making up the embodiments of theinvention described herein are referred to as operations, steps, ormodules. It will be recognized by one of ordinary skill in the art thatthese operations, steps, and modules may be implemented in software, infirmware, in special purpose digital logic, and any combination thereofwithout deviating from the spirit and scope of the present invention asrecited within the claims attached hereto. This software, firmware, orsimilar sequence of computer instructions may be encoded and stored uponcomputer readable storage medium and may also be encoded within acarrier-wave signal for transmission between computing devices.

FIG. 3 illustrates a hand-held computer having processing modules toimplement a pen-based user interface according to another embodiment ofthe present invention. The hand-held computer includes variousprocessing modules used to implement its functions. These processingmodules include a user interface module 301, a data item input module302, a data storage module 303, one or more application modules 304, acommunication module 305, and a utility module 306. The applicationmodules 304 typically include a personal information manager (PIM)having a date book, address book, notepad, and to do list. Theapplication modules 304 interact with the data storage module 303, dataitem input module 302, user interface module 301, communication module305, and utility module 306 to provide the desired functionality.

The adaptive pen mode selection processing is performed within the dataitem input module 302 utilizing a pen input module 323, a text inputmodule 322, and a data item editing module 321. The data item inputmodule 302 interacts with other user interface modules 301 to providethe interaction with the user when the user draws data items on thedisplay 104 of the hand held computer using a pen 102. The pen inputmodules 323 interpret pen strokes drawn on the display 104 of the handheld computer 101, including when the hand held computer 101 isoperating in a pen input mode. The text input modules 322 interpretinput commands to enter text data such as ASCII characters when the handheld computer 101 is operating in a text input mode. The data itemediting modules 321 interpret pen strokes and input commands to select,edit, and delete the data items stored within the hand held computer101. These modules 321–323 interact with other user interface modules301 to provide this functionality to the user of the hand held computer101.

FIG. 4 illustrates a hand-held computing system having multiple dataitems in which data items may be selected according to one embodiment ofthe present invention. Several text data items 401–420 are illustratedon the display 104 of the hand-held computer 101. When a user desires toedit one or more words within a text data item, the user selects theword and enters the changes. Typically, the hand-held computer operatesin one of several operating modes. These operating modes include a peninput mode, a text input mode, and a data item selection mode.

When the hand-held computer is operating in the pen input mode, theuser's pen strokes on the screen are interpreted as enteringcorresponding pen ink marks in a data window in which the pen stroke ismade. In this operating mode, the pen strokes create hand-written datasimilar to a hand-written pen stroke made upon a page. The data istypically captured as a sequence of graphic pen strokes; however, thedata may also be represented using bit-mapped graphics and othermechanisms for capturing and representing hand-written, user entereddata items.

When the hand-held computer is operating in the text input mode, a softinput window 431 appears upon the display 104 of the hand-held computer101 to allow a user to enter text characters into the computer 101. Thissoft input window 431 provides a user with an on-screen keyboard inwhich individual ASCII characters may be entered. The soft input window431 may also present a user with an input pen stroke drawing area inwhich individual characters or words may be entered and interpreted. TheASCII text input in this operating mode will appear as text on thedisplay 104 as if the text was entered using a standard keyboard.

When the hand-held computer is operating in the data item selectionmode, the user pen strokes made upon the display 104 are interpreted asan attempt to select a data item located beneath the pen stroke in orderto permit the data item to be edited or deleted. This operation issimilar to the operation of a desktop computer in which items areselected by dragging a cursor over the items using a pointing devicesuch as a mouse or trackball.

An example of the operation of a hand-held computer in both the peninput mode and the data item selection mode are illustrated in FIG. 4.Three identical text data items 401, 410, and 420 are positioned ondifferent lines of the display 104 of the hand-held computer. The secondtext data item 410 illustrates the result that occurs when a pen strokeis made across the word “Entered” when the hand-held computer isoperating in the pen input mode. The pen stroke results in the creationof a pen ink line 411 that is located on top of the text word. Thisresult is similar to the result when a user draws a line over a wordwhen both are printed on a sheet of paper.

The third text data item 420 illustrates the result that occurs when apen stroke is again made across the word “Entered” when the hand-heldcomputer is operating in the data item selection mode. In this mode, thepen stroke is interpreted as an instruction to select the word “Entered”421. The operation will result in the word “Entered” being 421 beingselected. The selected word 421 is displayed in a highlighted displaymode. Once the data item 421 is selected, it may be deleted, edited,cut, or replaced using a corresponding command.

When the hand-held computer is operating in an adaptive pen mode, theresult will be the same as shown in data items 420–421. This result isidentical to the hand-held computer operating in the data item selectionmode when the pen stroke is over an existing data item. Similarly, theresult would be the creation of an ink pen stroke 412 similar to theoperation in the pen input mode only when the pen stroke is not locatedover an existing data item. The operation of the adaptive pen modeselection processing causes the operating mode of the hand-held computer101 to correctly switch to the appropriate operating mode. The adaptivepen mode selection processing makes these determinations based upon thelocation of the pen stroke as well as other defining characteristics.These other characteristics may include, but are not limited to, thedirection of the pen stroke (L-to-R or R-to-L), the percentage of aword(s) over which the pen stroke is made, the placement of the penstroke within the word (top, middle, bottom), the age of the data itemover which the pen stroke is made, and characteristics of the data itemitself.

FIG. 5 illustrates a data item input module according to one possibleembodiment of the present invention. The data item input module 302interacts with one or more user interface modules 301 to accept theinput of data items, to permit existing data items to be altered anddeleted, and to pass these data items to various application modules 304for use in creating, editing, and displaying data within a window on ahand-held computer 101. The data item input modules comprises a textinput module 322 to permit text data to be entered into the hand-heldcomputer 101 and a pen input module 323 to permit pen ink data to beentered into the hand-held computer 101. The data item input modulemight also support other formats for data, such as the entry of audiodata using an audio input module 324. The other data formats wouldpossess a corresponding input module for accepting the input of the datafrom the user and processing the data into a format that may berepresented and stored within the hand-held computer. The data would berepresented upon the display of the hand-held computer at least as anicon that may be selected, edited and deleted in the same manner thatink and text data is manipulated herein.

The data item input module 302 also comprises a data item editing module321 that performs the selection processing, including the adaptive penmode selection processing described herein. The data item editing module321 comprises a data item position module 502, a pen position module503, and an input mode control module 504 that all interact with anadaptive pen mode selection module 501 to perform the data itemselection and editing operations.

The data item position module 502 determines the position of data itemsthat are located within a window within the hand-held computer 101.These positions are needed to determine whether a new pen stroke islocated over an existing data item, and if so, where within the existingdata item the pen stroke has been made. Similarly, the pen positionmodule 503 provides the position information for the pen stroke, as wellas its direction of movement. This information is used by the adaptivepen mode selection module 501 to determine if the pen stroke shouldcause the operating mode of the hand-held computer to switch to the dataitem selection mode.

The input mode control module 504 performs the operation necessary tomaintain the various operating modes as well as to detect instructionsto switch from one mode to another. Typically, an input control 120–122exists within the user interface to permit the user to indicate theoperating mode that he or she wishes the computer 101 to be operating.These input controls 120–122 may include selection buttons such as radiobuttons that a user can activate or deactivate as desired. These inputcontrols may also include other user interface input options asdescribed earlier.

The adaptive pen mode selection module 501 accepts data from the othermodules within the data item editing module 321 to determine whether themost recent pen stroke should be interpreted as an instruction to selectan existing data item regardless of the particular operating modecurrently selected for the hand-held computer. The particular operationperformed by the adaptive pen mode selection module 501 are bestillustrated using the particular examples shown in FIGS. 6 a, 6 b and 6c.

FIGS. 6 a, 6 b and 6 c illustrates various data items and correspondingpen strokes according to one possible embodiment of the presentinvention. FIG. 6 a illustrates a two-word data item 601 that comprisesa sequence of text data characters 611–619. Within each of the text datacharacters 611–619, particular character values may be associated withthe text data characters. By viewing the sequence of text datacharacters as an array containing character data, two words, “text”611–614 and “word” 616–619, are found that are separated by a spacecharacter 615, which is indicated by “_” character. Each of these textdata characters possess various properties. These properties include anx,y position within a window, an x,y size, a ASCII character value, andan indication of when the character was created or if the character isconsidered to be “persistent.” A character is persistent if thecharacter has existed for a period of time greater than apre-defined_time period. The property is intended to distinguish newlycreated data items, which may not be completely entered from data itemsthat have existed long enough to be considered complete.

FIG. 6 b illustrates the sequence of text data items 621–629 containingtwo words as discussed above along with two pen strokes made in themiddle of a word 630 and made below another word 631. Each pen strokepossesses properties that indicate its starting position, its endingposition, and any other position data needed to adequately characterizeand describe the pen stroke. This other data may include the slope ofthe pen stroke, the direction of pen stroke, and any intermediatesegment information necessary to describe the pen stroke. Typically apen stroke is described as a sequence of segments that approximatelinear lines having a constant slope. When the slope changes, a newsegment is created.

Considering the first of the two pen strokes 630–631 within FIG. 6 b,the pen stroke 631 is generally positioned at the middle of the textdata characters 626–629 containing “WORD.” The pen stroke has one endwithin the space text data item 625 and the other end within the “D”text data item 629. The pen stroke is a L-to-R pen stroke if it wasdrawn from the space 625 to the D 629. Similarly, pen stroke is a R-to-Lpen stroke if it was drawn from the D 629 to the space 625.

In contrast, consider the second of the two pen strokes 632 within FIG.6 b. This pen stroke 632 is positioned below the text in data characters621–624 containing “TEXT.” The pen stroke has one end within the first“T” text data item 621 and the other end within the second “T” text dataitem 624. The pen stroke is a L-to-R pen stroke if it was drawn from thefirst T 621 to the second T 624. Similarly, pen stroke is a R-to-L penstroke if it was drawn from the second T 624 to the first T 621.

The adaptive pen mode selection module 501 will use the properties ofthe data items 621–629 and the pen strokes 630–631 to determine whethera particular pen stroke should be treated as a selection instruction forthe data item(s) beneath the pen stroke or whether the pen stroke is tobe treated as an input pen stroke. If the pen stroke is to be treated asa selection instruction, the data items will be selected andhighlighted. If the pen stroke is to be treated as an input pen stroke,an ink line will be drawn where the pen stroke was located.

While the above example uses a sequence of text data characters, andthus the characters appear to possess a uniform size, this example maybe readily extended to other sequences. Such sequences include asequence of ink data items, a sequence of audio icons, a sequence ofvideo icon, or a combination of data items possessing the aboveproperties.

FIG. 6 c illustrates a combination of a pen ink data item with asequence of text data items. An ink data item 641 is shown next to asequence of text data items 642–645. A pen stroke 640 has one end withinthe ink data item 641 and another end within the “T” text character dataitem 645. The processing in FIG. 6 c is the same as discussed above inFIG. 6 b. Any combination of data items may be used without deviatingfrom the spirit and scope of the present invention as set forth in theclaims recited herein.

FIG. 7 illustrates an operational flow for adaptive pen mode selectionaccording to an exemplary preferred embodiment of the present invention.The operation of the hand-held computer begins at 701, and the Enter PenMode Module 702 is operative when the user instructs the computer 101 tooperate in the pen input mode using an appropriate control input. TheEnter Pen Mode Module 702 configures the operation of the user interfaceof the hand-held computer to operate in the pen input mode. In thisoperating mode, the movement of a pen 102 across the display 104 willcreate a pen stroke within the window shown on top of the display.

When a user presses the pen 102 onto the display 104, a buttonDown eventis triggered. The Capture Mouse Event Module 703 recognizes this eventand begins the operational flow for the adaptive pen mode selection. TheCapture Mouse Event Module 703 also captures mouseMove events andbuttonUp events. Once one of the events is captured, test operation 704checks to determine if the event is a buttonUp event. Typically, theprocessing begins with buttonDown event when the pen 102 is first placedonto the display 104 and continues until the occurrence of a buttonUpevent when the pen 102 is lifted from the display 104. The x,y locationof the buttonDown event corresponds to the first end of a pen stroke.Similarly, the x,y location of the buttonUp event corresponds to thesecond end of the pen stroke. The sequence of mouseMove events definesthe movement of the pen 102 across the display that defines thedirection and length of the pen stroke.

If the event is not a buttonUp event, the operation proceeds to testoperation 705. In test operation 705, the x,y location of the event ischecked to see if it is located over a text data item. If the event islocated over a text data item, the adaptive pen mode selection assumesthat the user wants to select the text item rather than draw a penstroke over the text item. As such, the operational flow continues to aSwitch Pen Mode Module 706. The Switch Pen Mode Module 706 temporarilyplaces the operation of the computer 101 into an input select mode fromany other operating mode and selects the text data item. The hand-heldcomputer 101 will continue in this operating mode, and will continue inselecting additional data items beneath the pen stroke, until a buttonUpevent is detected. The additional data items are selected within theSelect Until buttonUp Module 707, with the Module 707 maintaining a listof all currently selected items. Once the latest data item has beenincluded within the list of selected data items, the operational flowreturns to the Capture mouseEvent module 703 to obtain the next mouseevent until a buttonUp event is detected.

Returning to test operation 705 if the event tested is not located overa text data item, the operational flow continues to a Normal Ink Entrymodule 711. In this module 711, the event is processed as if the pen isintending to enter an ink pen stroke onto the window of the hand-heldcomputer 101. The operational flow continues to determine if theadaptive pen mode selection should switch from an ink pen mode to a dataitem selection mode based upon the characteristics of the pen stroke.This determination continues with test operation 712 to determine if thenewly entered pen stroke is now over an existing data item. If thestroke is not over an existing data item, the operation flow branchesNO, and the processing ends at 709 as the pen stroke is being made on anempty portion of the window.

If test operation 712 determines that the newly entered pen stroke islocated over an existing data item, test operation 713 is used todetermine if the existing data item is considered persistent. Apersistent data item is a data item that has been present within thecomputer 101 for greater than a predetermined period of time. In apreferred embodiment, the predetermined period of time is 2 seconds. Theassumption is that user will be adding additional hand-written penstrokes to a data item within this short period of time. For example, auser may dot an “i” or cross a “t” shortly after the character itselfhas been printed; however, the user will be selecting the “i” or “t” ifthe character was written a few days before. If test operation 713determines that the existing data item is not persistent, the processingends at 709 as the assumption is made that the latest pen stroke is partof a newly created data item.

If test operation 713 determines that the existing data item ispersistent, test operation 714 will check to see if the pen strokepossesses a drag direction indicating a selection action. As wasdiscussed above, pen strokes possess properties including a dragdirection indicating if the pen stroke has been made in a left-to-rightor a right-to-left direction. The adaptive pen mode selection maydistinguish selection operations over existing data items by requiringthe pen strokes to occur in one of the two directions with the otherdirection being used to place a pen stroke over an existing data item.Typically, a system parameter is set to indicate the direction of theselection action. If the drag direction of the pen stroke is not thesame as the selection action, the processing ends at 709 as the penstroke is interpreted as being a stroke operation and not a selectionoperation. If the drag direction of the pen stroke is the same as theselection action, the processing continues to test operation 715.

Test operation 715 checks to see if the pen stroke is located within aselection location for a particular data item. As was discussed abovewith regards to FIG. 6, a pen stroke may be made over data items nearthe top, in the middle, and near the bottom of the individual dataitems. These locations generally correspond to above, in the middle, andbelow meaningful information within data items. An assumption is madethat an item is to be selected only when the pen stroke is made “in themiddle” of the data item. The selection location is used to define aregion of the data item through which a pen stroke is made that will beinterpreted as being “in the middle.” A system parameter may specify therange of locations about its center, which may be interpreted as being“in the middle.” This range may be as narrow or wide as desired by auser. If test operation 715 determines that the pen stroke is not withinthe selection location, the processing ends at 709, as the pen stroke isnot in the middle of the data item.

If test operation 715 determines that the pen stroke is within theselection location, test operation 716 is used to determine if “most” ofan item has been “selected.” As the pen stroke moves across a data itemthrough its selection location, a determination needs to be made whetherthe item in question should be selected as part of the currentoperation. In FIG. 6 b, one end of the pen stroke began within the spacetext character 626 and the other end of the pen stroke terminated withinthe D text character 629. If the data items are considered to beseparate characters, the adaptive pen mode selection determines whetherthese end characters should be included within the list of selecteditems. This determination may be made by checking to see if the penstroke has been made over more than 50% of the data item. If so the dataitem should be selected. For text data items, a sequence of text dataitems may check for the existence of words separated by spaces and/orpunctuation characters. In such a case, the entire word may be selectedwhen more than 50% of its characters have been covered by a pen stroke.FIG. 6 b illustrates this operation as the pen stroke 640 begins withinthe ink pen data item 641 and then covers all of the adjoining text dataitems 642–645. Because the pen stroke 640 does not cover a significantportion of the ink pen data item 641, the pen stroke 640 selects justthe text data items 642–645.

If test operation 716 determines that most of a data item is not coveredby the pen stroke, the processing ends at 709. If test operation 716determines that most of a data item is covered by the pen stroke, theprocessing continues with the Switch Pen Mode Selection module 706 asdiscussed above. The data item will be selected and the processing willcontinue in the select pen mode until a mouseUp event is detected.

Returning to test operation 704 if the captured mouseEvent is a mouseUp,the pen stroke is considered ended. If test operation 704 determinesthat the captured mouseEvent is a mouseUp, the processing continues to aReturn Pen Mode module 708 to set the operating mode back to itsoriginal user specified operating mode. If the operating mode had beenswitched to the input pen selection mode from another operating mode, anitem is selected and highlighted; however, the pen should be returned toits user selected operating mode. If the operating mode had not beenswitched to the input pen selection mode from another operating mode,the Return Pen Mode module 708 performs no operation. If the operatingmode was originally in the input pen selection mode, the Return Pen Modemodule 708 also performs no operation. Now that the operating mode hasbeen returned to its original mode, the user may enter another penstroke if desired. If one or more data items have been selected, thedata items will appear as highlighted items that may be cut, deleted orreplaced with one or more data items using a paste command. Theseadditional editing operations are not part of the adaptive pen modeselection operations.

While the preferred embodiment includes the sequence of test operations712–716 to determine whether a pen stroke drawn over a data item shouldcause the data item to be selected, one skilled in the art willunderstand that other embodiments of the present invention may includeonly some, but not all, of these test operations 712–716 in making thisdetermination.

The present invention is presently embodied as a method, apparatus, andcomputer program product, such as a computer storage medium orpropagated signal containing a computer program, for providing anadaptive pen mode selection function as part of a data input portion ofa user interface within a hand-held computer. While the embodimentsdescribed above describe a hand-held computer for its operation, thepresent invention, as recited within the claims attached hereto,includes any computing system that utilizes a pointing device to enterink data items and text data items as discussed hereon. While theinvention has been particularly shown and described with reference topreferred embodiments thereof, it will be understood by those skilled inthe art that various other changes in the form and details may be madetherein without departing form the spirit and scope of the invention.

1. A method of providing an adaptive pen mode selection within a peninput based computing system having one or more user selected operatingmodes including an ink pen input mode, a text input mode, and a dataitem selection mode, the method comprising: detecting a pen strokehaving a location within a window on a display of the computing system,the pen stroke possesses properties including the location including astarting location, and ending location, a direction, and one or moresegments of constant slope; determining a location of all existing dataitems displayed within the window containing the location of the penpress, the existing data items possess properties including a data type,a position, a size, and a selection location, and a persistence age; ifthe computer is operating in the data item selection mode, selecting alldata items having location that corresponds to the location of the penstroke; if the computer is not operating in the data item selectionmode, perform the following: determining whether the user desired toselect all data items having location that corresponds to the locationof the pen stroke using one or more of the properties associated withthe pen stroke and the existing data items; and selecting all data itemsdetermined to be intended to be selected by the pen stroke.
 2. Themethod according to claim 1, wherein the pen stroke possesses propertiesincluding the location including a starting location, and endinglocation, a direction, and one or more segments of constant slope; andthe properties of the pen stroke are used in determining whether theuser intends to select all data items having location that correspondsto the location of the pen stroke.
 3. The method according to claim 2,wherein the existing data items possess properties including a datatype, a position, a size, and a selection location, and a persistenceage; and the properties of the existing data items are used indetermining whether the user intends to select all data items havinglocation that corresponds to the location of the pen stroke.
 4. Themethod according to claim 1, wherein the selecting all data itemsdetermined to be intended to be selected by the pen stroke furthercomprises: switching the computing system to the data item selectionmode; selecting all data items identified to correspond to the penstroke; and switching the computing system back to the previous userselected operating mode.
 5. The method according to claim 1, wherein theselection location for the data item comprises a selection location, aselection size, and a selection direction; a pen stroke is within theselection location when the pen stroke location corresponds to alocation within the area defined by the selection location, theselection size, and the selection direction.
 6. The method according toclaim 5, wherein the determining whether the user intends to select alldata items having location that corresponds to the location of the penstroke further comprises: determining if direction of the pen stroke isin a selection direction; determining if the location of the pen strokeis within the selection location of all data items; and if direction ofthe pen stroke is in a selection direction, selecting all data items inwhich the pen stroke is within the selection location.
 7. The methodaccording to claim 5, wherein the selecting all data items determined tobe intended to be selected by the pen stroke further comprises:determining if direction of the pen stroke is in a selection direction;determining if the location of the pen stroke is within the selectionlocation of all data items; determining if the persistent age of alldata items is greater than a predetermined period; determining thelength of the pen stroke across each data item; and if direction of thepen stroke is in a selection direction, selecting all data items inwhich the pen stroke is within the selection location when thepersistent age is greater than the predetermined period and when thelength of the pen stroke across a data item is greater than 50% of thesize of the data item in the direction of the selection direction. 8.The method according to claim 7, wherein the predetermined period is 2seconds and the selection direction is left-to-right.
 9. The methodaccording to claim 7, wherein the selection location corresponds to themiddle of the vertical side of a data item; and the selection sizecorresponds to 25 to 50% of the size of the data item centered aroundthe selection location.
 10. A computer program product readable by acomputing system and encoding a set of computer instructions forproviding an adaptive pen mode selection within a pen input basedcomputing system having one or more user selected operating modeincluding an ink pen input mode, a text input mode, and a data itemselection mode, comprising: detecting a pen stroke having a locationwithin a window on a display of the computing system, the pen strokepossesses properties including the location including a startinglocation, and ending location, a direction, and one or more segments ofconstant slope; determining a location of all existing data itemsdisplayed within the window containing the location of the pen press,the existing data items possess properties including a data type, aposition, a size, and a selection location, and a persistence age; ifthe computer is operating in the data item selection mode, selecting alldata items having location that corresponds to the location of the penstroke; if the computer is not operating in the data item selectionmode, perform the following: determining whether the user desired toselect all data items having location that corresponds to the locationof the pen stroke using the direction property of the pen stroke and thepersistence age of the existing data items; and selecting all data itemsdetermined to be intended to be selected by the pen stroke; wherein thepen stroke possesses properties including the location including astarting location, and ending location, a direction, and one or moresegments of constant slope; the properties of the pen stroke are used indetermining whether the user intends to select all data items havinglocation that corresponds to the location of the pen stroke; theexisting data items possess properties including a data type, aposition, a size, and a selection location, and a persistence age; andthe properties of the existing data items are used in determiningwhether the user intends to select all data items having location thatcorresponds to the location of the pen stroke.
 11. The computer programproduct according to claim 10, wherein the selecting all data itemsdetermined to be intended to be selected by the pen stroke furthercomprises: switching the computing system to the data item selectionmode; selecting all data items identified to correspond to the penstroke; and switching the computing system back to the previous userselected operating mode.
 12. The computer program product according toclaim 10, wherein the selection location for the data item comprises aselection location, a selection size, and a selection direction; a penstroke is within the selection location when the pen stroke locationcorresponds to a location within the area defined by the selectionlocation, the selection size, and the selection direction.
 13. Thecomputer program product according to claim 12, wherein the determiningwhether the user intends to select all data items having location thatcorresponds to the location of the pen stroke further comprises:determining if direction of the pen stroke is in a selection direction;determining if the location of the pen stroke is within the selectionlocation of all data items; and if direction of the pen stroke is in aselection direction, selecting all data items in which the pen stroke iswithin the selection location.
 14. The computer program productaccording to claim 12, wherein the selecting all data items determinedto be intended to be selected by the pen stroke further comprises:determining if direction of the pen stroke is in a selection direction;determining if the location of the pen stroke is within the selectionlocation of all data items; determining if the persistent age of alldata items is greater than a predetermined period; determining thelength of the pen stroke across each data item; and if direction of thepen stroke is in a selection direction, selecting all data items inwhich the pen stroke is within the selection location when thepersistent age is greater than the predetermined period and when thelength of the pen stroke across a data item is greater than 50% of thesize of the data item in the direction of the selection direction. 15.The computer program product according to claim 14, wherein thepredetermined period is 2 seconds and the selection direction isleft-to-right.
 16. The computer program product according to claim 14,wherein the selection location corresponds to the middle of the verticalside of a data item; and the selection size corresponds to 25% of thesize of the data item centered around the selection location.
 17. A peninput based computing system having adaptive pen mode selection withinone or more user selected operating mode including an ink pen inputmode, a text input mode, and a data item selection mode, the computingsystem comprising: a programmable processor; a user display configuredto generate pen strokes corresponding to user created pen movementsacross the display; one or more data items having properties including adata type, a position, a size, and a selection location, and apersistence age; an user interface module for capturing user generatedpen strokes; and a data item input module for determining whether dataitems are to selected by a pen stroke; wherein the pen stroke possessesproperties including the location including a starting location, andending location, a direction, and one or more segments of constantslope; and the data item input module determines whether the userintends to select all data items having location that corresponds to thelocation of the pen stroke using the direction property of the penstroke and the persistence age of the existing data items and selectsall data items determined to be intended to be selected by the penstroke.
 18. The computing system according to claim 17, wherein theproperties of the pen stroke are used in determining whether the userintends to select all data items having location that corresponds to thelocation of the pen stroke.
 19. The computing system according to claim18, wherein the properties of the existing data items are also used whenthe data item input module determines whether the user intends to selectall data items having location that corresponds to the location of thepen stroke.
 20. The computing system according to claim 18, wherein thedata item input module selects all data items determined to be intendedto be selected by the pen stroke by performing the following: switchingthe computing system to the data item selection mode; selecting all dataitems identified to correspond to the pen stroke; and switching thecomputing system back to the previous user selected operating mode. 21.The computing system according to claim 20, wherein the selectionlocation for the data item comprises a selection location, a selectionsize, and a selection direction; a pen stroke is within the selectionlocation when the pen stroke location corresponds to a location withinthe area defined by the selection location, the selection size, and theselection direction.
 22. The computing system according to claim 21,wherein the data item input module determines whether the user intendsto select all data items having location that corresponds to thelocation of the pen stroke further comprises: determining if directionof the pen stroke is in a selection direction; determining if thelocation of the pen stroke is within the selection location of all dataitems; and if direction of the pen stroke is in a selection direction,selecting all data items in which the pen stroke is within the selectionlocation.
 23. The computing system according to claim 21, wherein thedata item input module selects all data items determined to be intendedto be selected by performing the following: determining if direction ofthe pen stroke is in a selection direction; determining if the locationof the pen stroke is within the selection location of all data items;determining if the persistent age of all data items is greater than apredetermined period; determining the length of the pen stroke acrosseach data item; and if direction of the pen stroke is in a selectiondirection, selecting all data items in which the pen stroke is withinthe selection location when the persistent age is greater than thepredetermined period and when the length of the pen stroke across a dataitem is greater than 50% of the size of the data item in the directionof the selection direction.
 24. The computing system according to claim23, wherein the predetermined period is 2 seconds and the selectiondirection is left-to-right.
 25. The computing system according to claim23, wherein the selection location corresponds to the middle of thevertical side of a data item; and the selection size corresponds to 25%of the size of the data item centered around the selection location.